Field of the Invention
The present invention relates to video coding. In particular, the present invention relates to coding techniques associated with simplified chroma intra prediction based on reconstructed luma and chroma pixels.
Description of the Related Art
Motion compensated inter-frame coding has been widely adopted in various coding standards, such as MPEG-1/2/4 and H.261/H.263/H.264/AVC. While motion-compensated inter-frame coding can effectively reduce bitrate for compressed video, intra coding is required to compress the regions with high motion or scene changes. Besides, intra coding is also used to process an initial picture or to periodically insert I-pictures or I-blocks for random access or for alleviation of error propagation. Intra prediction exploits the spatial correlation within a picture or within a picture region. In practice, a picture or a picture region is divided into blocks and the intra prediction is performed on a block basis. Intra prediction for a current block can rely on pixels in neighboring blocks that have been processed. For example, if blocks in a picture or picture region are processed row by row first from left to right and then from top to bottom, neighboring blocks on the top and neighboring blocks on the left of the current block can be used to form intra prediction for pixels in the current block. While any pixels in the processed neighboring blocks can be used for intra predictor of pixels in the current block, very often only pixels of the neighboring blocks that are adjacent to the current block boundaries on the top and on the left are used.
The intra predictor is usually designed to exploit spatial features in the picture such as smooth area (DC mode), vertical line or edge, horizontal line or edge and diagonal line or edge. Furthermore, correlation often exists between the luminance (luma) and chrominance (chroma) components. Therefore, reconstructed luma pixels can be used to derive the intra chroma prediction. In recent development of High Efficiency Video Coding (HEVC), a chroma intra prediction method based on co-located reconstructed luma blocks has been disclosed. The type of chroma intra prediction is termed as LM prediction or LM mode. The main concept is to use the reconstructed luma pixels to generate the predictors of corresponding chroma pixels. FIG. 1A and FIG. 1B illustrate the prediction procedure. First, the neighboring reconstructed pixels of a co-located luma block in FIG. 1A and the neighboring reconstructed pixels of a chroma block in FIG. 1B are used to derive the correlation parameters between the blocks. Then, the predicted pixels of the chroma block are generated using the parameters and the reconstructed pixels of the luma block. In the parameters derivation, the first above reconstructed pixel row and the second left reconstructed pixel column of the current luma block are used. The specific row and column of the luma block are used in order to match the 4:2:0 sampling format of the chroma components. The following illustration is based on 4:2:0 sampling format. LM-mode chroma intra prediction for other sampling formats can be derived similarly.
In the Test Model Version 5.0 (HM-5.0), the LM mode is applied to predict chroma samples based on a linear model using reconstructed luma samples of the co-located prediction unit (PU). The parameters of the linear model consist of slope (a>>k) and y-intercept (b), where “>>” corresponds to the right shift operation. The parameters are derived based on the neighboring luma and chroma samples according to a least mean square criterion. The prediction sample, predSamples[x,y] for the chroma sample to be coded in the LM mode is derived as follows, where x,y=0 . . . nS−1 and nS corresponds to the block size.
First, variable k3 and the sample array pY′ are derived as:k3=Max(0,BitDepthC+Log 2(nS)−14),  (1)where BitDepthC denotes the internal chroma bit depth (i.e., the bit depth with which the chroma signal is processed during video coding process), andpY′[x,−1]=(PLM[2x−1,−1]+2*PLM[2x,−1]+PLM[2x+1,−1]+2)>>2,  (2)pY′[−1,y]=(PLM[−1,2y]+PLM[−1,2y+1])>>1,  (3)pY′[x,y]=(recSamplesL[2x,2y]+recSamplesL[2x,2y+1])>>1,  (4)where x=0 . . . nS−1, PLM[x,y] denotes the neighboring reconstructed luma samples, and recSamplesL[x,y] denotes the current reconstructed luma samples of the co-located luma block. The sample array pY′ are derived from reconstructed luma samples. Accordingly, pY′ is also called derived co-located luma sample in this disclosure. In equations (2) through (4), pY′[x,y] is only evaluated at positions co-located with the chroma samples.
In HM-5.0, the characteristics of the neighboring reconstructed luma samples and neighboring reconstructed chroma samples of the current block are used to determine the linear-model parameters a, k, and b for LM-mode chroma intra prediction. The derived co-located luma pixels of the current luma block can be derived from the current reconstructed luma pixels of the current luma block at pixel locations co-located with the chroma pixels of the current chroma block.
To exemplify the linear model relating the chroma intra prediction with the derived co-located luma pixels, a set of variable, including L, C, LL, LC and k2, are defined. Variables L, C, LL, LC and k2 are derived as follows.
                                              ⁢                              L            =                          (                                                                    ∑                                          y                      =                      0                                                              nS                      -                      1                                                        ⁢                                                            p                      Y                      ′                                        ⁡                                          [                                                                        -                          1                                                ,                        y                                            ]                                                                      +                                                      ∑                                          x                      =                      0                                                              nS                      -                      1                                                        ⁢                                                            p                      Y                      ′                                        ⁡                                          [                                              x                        ,                                                  -                          1                                                                    ]                                                                                  )                                >>                      k            ⁢                                                  ⁢            3                                              (        5        )                                                          ⁢                              C            =                          (                                                                    ∑                                          y                      =                      0                                                              nS                      -                      1                                                        ⁢                                      p                    ⁡                                          [                                                                        -                          1                                                ,                        y                                            ]                                                                      +                                                      ∑                                          x                      =                      0                                                              nS                      -                      1                                                        ⁢                                      p                    ⁡                                          [                                              x                        ,                                                  -                          1                                                                    ]                                                                                  )                                >>                      k            ⁢                                                  ⁢            3                                              (        6        )                                                          ⁢                              LL            =                          (                                                                    ∑                                          y                      =                      0                                                              nS                      -                      1                                                        ⁢                                                                                    p                        Y                        ′                                            ⁡                                              [                                                                              -                            1                                                    ,                          y                                                ]                                                              2                                                  +                                                      ∑                                          x                      =                      0                                                              nS                      -                      1                                                        ⁢                                                                                    p                        Y                        ′                                            ⁡                                              [                                                  x                          ,                                                      -                            1                                                                          ]                                                              2                                                              )                                >>                      k            ⁢            3                                              (        7        )                                          LC          =                      (                                                            ∑                                      y                    =                    0                                                        nS                    -                    1                                                  ⁢                                                                            p                      Y                      ′                                        ⁡                                          [                                                                        -                          1                                                ,                        y                                            ]                                                        *                                      p                    ⁡                                          [                                                                        -                          1                                                ,                        y                                            ]                                                                                  +                                                ∑                                      x                    =                    0                                                        nS                    -                    1                                                  ⁢                                                                            p                      Y                      ′                                        ⁡                                          [                                              x                        ,                                                  -                          1                                                                    ]                                                        *                                      p                    ⁡                                          [                                              x                        ,                                                  -                          1                                                                    ]                                                                                            )                          >>                  k          ⁢          3                                    (        8        )                                                          ⁢                              k            ⁢                                                  ⁢            2                    =                      Log            ⁢                                                  ⁢            2            ⁢                          (                                                (                                      2                    *                    nS                                    )                                >>                                  k                  ⁢                  3                                            )                                                          (        9        )            
As shown in equations (5) through (8), L corresponds to the sum of reconstructed luma samples in the neighboring area of the current block, C corresponds to the sum of reconstructed chroma samples in the neighboring area of the current block, LL corresponds to the sum of squared reconstructed luma samples in the neighboring area of the current block, LC corresponds to the sum of cross-product of reconstructed luma samples and reconstructed chroma samples in the neighboring area of the current block. Furthermore, L, C, LL, and LC are right shifted by k3 bits to take into account of the internal bit depth with which the chroma signal is processed during video coding process (i.e., bitDepthC) and the block size (i.e., nS).
The linear-model parameters a, b and k are derived as follows.a1=(LC<<k2)−L*C,  (10)a2=(LL<<k2)−L*L,  (11)k1=Max(0,Log 2(abs(a2))−5)−Max(0,Log 2(abs(a1))−14)+2,  (12)a1s=a1>>Max(0,Log 2(abs(a1))−14),  (13)a2s=abs(a2>>Max(0,Log 2(abs(a2))−5)),  (14)a3=a2s<1?0:Clip3(−215,215−1,(a1s*lmDiv+(1<<(k1−1)))>>k1),  (15)a=a3>>Max(0,Log 2(abs(a3))−6),  (16)k=13−Max(0,Log 2(abs(a3))−6), and  (17)b=(L−((a*C)>>k1)+(1<<(k2−1)))>>k2,  (18)where lmDiv is specified in Table 1 for all a2s values.
Parameter a1 as defined in equation (10) corresponds to a covariance-like value associated with the neighboring reconstructed luma pixels of the current luma block and the neighboring reconstructed chroma pixels of the current chroma block. The covariance α(X,Y) associated with random variables X and Y are defined as σ(X,Y)=E[XY]−E[X]E[Y], where E[.] is the expected value of the underlying random variable. For a random variable with a uniform distribution, the expected value is equivalent to the average value. Variables L, C, LL, and LC as shown in equations (5) through 8 are right shifted by k3 bits, where k3 is related to the block size (i.e., nS). In other words, if the neighboring reconstructed luma pixels of the current luma block are considered as a first uniformly distributed random variable (i.e., X) and the neighboring reconstructed chroma pixels of the current chroma block are considered as a second uniformly distributed random variable (i.e., Y), a1 has a form similar to a covariance value for uniformly distributed random variables X and Y. In equation (10), LC is left shifted by k2 bits in order to match the scaling of L*C. Accordingly, a1 as defined in equation (10) has a covariance-like form associated with the neighboring reconstructed luma pixels of the current luma block and the neighboring reconstructed chroma pixels of the current chroma block. The covariance-like value may be scaled to a desired range and the scaling may be performed by left or right shifting the covariance-like value. For example, the scaling of the covariance-like value used in HM-5.0 is shown in equation (13).
Similarly, parameter a2 as defined in equation (11) corresponds to a variance-like value associated with the neighboring reconstructed luma pixels of the current luma block. The variance-like value may be scaled to a desired range and the scaling may be performed by left or right shifting the variance-like value. For example, the scaling of the variance-like value used in HM-5.0 is shown in equation (14). A division factor, lmDiv is then determined by dividing with rounding a first data range by a2s. In HM-5.0, the division with rounding by a2s is implemented using a look-up table as shown in Table 1. In HM-5.0, the first data range corresponds to 2^15 (i.e., 215). Accordingly, lmDiv=(2^15+a2s/2)/a2s. An intermediate parameter, a3 is then determined according to a1s*lmDiv, where the product a1s*lmDiv is divided with rounding by 2k1 and the result is clipped between −215 and 215−1 if a2s≧1. If a2s is less than 1, a3 is set to 0. In HM-5.0, derivation of a3 is shown in equation (15), where the division with rounding by 2k1 is implemented by right shifting. The right shifting in equation (15) is performed to reverse the shifting operations that are applied to a1 and a2 in equation (13) and equation (14).
Parameters a and k for the linear model are then determined based on a3 as shown in equations (16) and (17) respectively. The y-intercept, b is determined according to equation (18). Finally, the value of the prediction samples predSamples[x,y] is derived as:predSamples[x,y]=Clip1C(((pY′[x,y]*a)>>k)+b),  (19)where x, y=0 . . . nS−1 andClip1C(w)=Clip3(0,(1<<BitDepthC)−1,w)clip3(t,u,v)=((v<t)?t:((v>u)?u:v))
TABLE 1a2s12345678lmDiv32768163841092381926554546146814096a2s910111213141516lmDiv36413277297927312521234121852048a2s1718192021222324lmDiv19281820172516381560148914251365a2s2526272829303132lmDiv13111260121411701130109210571024a2s3334353637383940lmDiv993964936910886862840819a2s4142434445464748lmDiv799780762745728712697683a2s4950515253545556lmDiv669655643630618607596585a2s5758596061626364lmDiv575565555546537529520512
As shown above, the derivation of the prediction sample, predSamples[x,y] is very computationally intensive. Not only it involves a large number of computations, it also requires buffer to store the table. Furthermore, some operations may require higher precision. For example, in equation (15), a1s is a signed 15-bit integer, therefore, a1s*lmDiv requires a 15-bit multiplier. This large multiplier introduces higher computational complexity. Therefore, it is desirable to simplify the derivation of the prediction sample.